This patent application is co-pending with one related patent application entitled xe2x80x9cMETHOD OF IMPROVING A DIGITAL IMAGE HAVING WHITE ZONESxe2x80x9d, filed on the same date by the same inventors and owned by the same assignees as this patent application.
The invention relates generally to image processing, and more particularly to a method of improving a digital image that adjusts to the image""s dynamic range.
When compared to the direct observation of scenes, color images in general have two major limitations due to scene lighting conditions. First, the images captured and displayed by photographic and electronic cameras suffer from a comparative loss of detail and color in shadowed zones. This is known as the dynamic range problem. Second, the images are subject to color distortions when the spectral distribution of the illuminant changes. This is known as the color constancy problem. (Note that for non-color imaging including non-optical imaging, the problem becomes simpler and is largely one of dynamic range compression, i.e., the capture and representation of detail and lightness values across wide ranging average signal levels that can vary dramatically across a scene.)
Electronic cameras (e.g., cameras based on CCD detector arrays, CMOS technology, etc.) are capable of acquiring image data across a wide dynamic range. This range is suitable for handling most illumination variations within scenes, and lens aperture changes are usually employed to encompass scene-to-scene illumination variations. Typically though, this dynamic range is lost when the image is digitized or when the much narrower dynamic range of print and display media are encountered. For example, most consumer-type images are digitized to 8-bits/color band (256 gray levels/color band) and most display and print media are even more limited to a 50:1 dynamic range.
A commonly encountered instance of the color constancy problem is the spectral difference between daylight and artificial (e.g., tungsten) light which is sufficiently strong to require photographers to shift to some combination of film, filters and processing to compensate for the spectral shift in illumination. Though film photographers can attempt to approximately match film type to spectral changes in lighting conditions, digital cameras must rely strictly on filters. However, these methods of compensation do not provide any dynamic range compression thereby causing detail and color in shadows to be lost or severely attenuated compared to what a human observer would actually see.
Another problem encountered in color and non-color image processing is known as color/lightness rendition. This problem results from trying to match the processed image with what is observed and consists of 1) lightness and color xe2x80x9chaloxe2x80x9d artifacts that are especially prominent where large uniform regions of an image abut to form a high contrast edge with xe2x80x9cgrayingxe2x80x9d in the large uniform zones, and 2) global violations of the gray world assumption (e.g., an all-red scene) which results in a global xe2x80x9cgraying outxe2x80x9d of the image.
Since human vision does not suffer from these various imaging drawbacks, it is reasonable to attempt to model machine vision based on human vision. A theory of human vision centered on the concept of a center/surround retinex was introduced by Edwin Land in xe2x80x9cAn Alternative Technique for the Computation of the Designator in the Retinex Theory of Color Vision,xe2x80x9d Proceedings of the National Academy of Science, Volume 83, pp. 3078-3080, 1986. Land drew upon his earlier retinex concepts disclosed in xe2x80x9cColor Vision and The Natural Image,xe2x80x9d Proceedings of the National Academy of Science, Volume 45, pp. 115-129, 1959, but harmonized these with certain findings of the neurophysiology of vision. All of the retinex concepts were intended to be models for human color perception. The earlier retinex concepts involved xe2x80x9crandom walksxe2x80x9d across image space and the resetting of the computation when color boundaries were crossed. Land""s 1986 retinex concept of human vision was proposed as a center/surround spatial computation where the center was 2-4 arc-minutes in diameter and the surround was an inverse square function with a diameter of about 200-250 times that of the center.
The application of Land""s human vision theories to image processing has been attempted in the prior art. For example, to mimic the dynamic range compression of human vision, a detector array with integrated processing in analog VLSI silicon chips used a logarithm transformation prior to the surround formation. See xe2x80x9cAnalog VLSI and Neural Systems,xe2x80x9d C. Mead, Addison-Wesley, Reading, Mass., 1989. In an attempt to improve color constancy, the implementation of a color retinex in analog VLSI technology is suggested by Moore et al., in xe2x80x9cA Real-time Neural System for Color Constancy,xe2x80x9d IEEE Transactions on Neural Networks, Volume 2, pp. 237-247, March 1991. In Moore et al., the surround function was an exponential and final processing before display of the image required the use of a variable gain adjustment that set itself by finding the absolute maximum and minimum across all three color bands"" signal values. However, none of the above-described prior art provided an image processing technique that could simultaneously accomplish/improve dynamic range compression, color independence from the spectral distribution of the scene illuminant, and color/lightness rendition.
To address these issues, U.S. Pat. No. 5,991,456 discloses a method of improving a digital image in which the image is initially represented by digital data indexed to represent positions on a display. The digital data is indicative of an intensity value Ii(x,y) for each position (x,y) in each i-th spectral band. The intensity value for each position in each i-th spectral band is adjusted to generate an adjusted intensity value for each position in each i-th spectral band in accordance with             ∑              n        =        1            N        ⁢                  W        n            ⁡              (                              log            ⁢                          xe2x80x83                        ⁢                                          I                i                            ⁡                              (                                  x                  ,                  y                                )                                              -                      log            ⁡                          [                                                                    I                    i                                    ⁢                                      (                                          x                      ,                      y                                        )                                                  *                                                      F                    n                                    ⁢                                      (                                          x                      ,                      y                                        )                                                              ]                                      )              ,      i    =    1    ,  …  ⁢      xe2x80x83    ,  S
where Wn is a weighting factor, xe2x80x9c*xe2x80x9d is the convolution operator and S is the total number of unique spectral bands. For each n, the function Fn(x,y) is a unique surround function applied to each position (x,y) and N is the total number of unique surround functions. Each unique surround function is scaled to improve some aspect of the digital image, e.g., dynamic range compression, color constancy, and lightness rendition. The adjusted intensity value for each position in each i-th spectral band is filtered with a common function. The improved digital image can then be displayed and is based on the adjusted intensity value for each i-th spectral band so-filtered for each position. For color images, a color restoration step can be added to give the image true-to-life color that closely matches human observation.
While this patented method performs well for scenes/images having widely varying lighting, reflectance and/or topographic features (referred to hereinafter as wide dynamic range images), the method provides a lesser degree of improvement for scenes/images having constrained lighting, reflectance and/or minimal topographic variations (referred to hereinafter as narrow dynamic range images). Furthermore, it has been found that the use of this patented method can cause large xe2x80x9cwhitexe2x80x9d zones in digital images to be xe2x80x9cgrayedxe2x80x9d. The larger and more constant the white zone, the greater the degree of graying. Such white zones are commonly found in artificial images generated by both computer graphics and document imaging applications.
Accordingly, it is an object of the present invention to provide a method of improving an image created with digital data for both color and non-color images.
Another object of the present invention to provide a method of improving a digital image in terms of the image""s dynamic range compression, color independence from the spectral distribution of the scene illuminant, and color/lightness rendition.
Still another object of the present invention to provide a method of improving a digital image so that the image appears similar to what is perceived by human vision in all kinds and levels of lighting across the entire scene.
Yet another object of the present invention is to provide a method of improving a digital image regardless of the image""s dynamic range.
A further object of the present invention is to provide a method of reducing the xe2x80x9cgrayingxe2x80x9d of large white zones in a processed digital image.
Other objects and advantages of the present invention will become more obvious hereinafter in the specification and drawings.
In accordance with the present invention, a method of processing a digital image is provided. The image is initially represented by digital data indexed to represent positions on a display. The digital data is indicative of an intensity value Ii(x,y) for each position (x,y) in each i-th spectral band. A classification of the image based on its dynamic range is then defined in each of the image""s spectral bands. The intensity value for each position in each i-th spectral band is adjusted to generate an adjusted intensity value for each position in each i-th spectral band in accordance with             ∑              n        =        1            N        ⁢                  W        n            ⁡              (                              log            ⁢                          xe2x80x83                        ⁢                                          I                i                            ⁡                              (                                  x                  ,                  y                                )                                              -                      log            ⁡                          [                                                                    I                    i                                    ⁢                                      (                                          x                      ,                      y                                        )                                                  *                                                      F                    n                                    ⁢                                      (                                          x                      ,                      y                                        )                                                              ]                                      )              ,      i    =    1    ,  …  ⁢      xe2x80x83    ,  S
where Wn is a weighting factor, xe2x80x9c*xe2x80x9d is the convolution operator and S is the total number of unique spectral bands. For each n, the function Fn(x,y) is a unique surround function applied to each position (x,y) and N is the total number of unique surround functions. Each unique surround function is scaled to improve some aspect of the digital image, e.g., dynamic range compression, color constancy, and lightness rendition. The adjusted intensity value for each position in each spectral band of the image is filtered with a filter function that is based on the dynamic range classification of the image. As a result, a filtered intensity value Ri(x,y) is defined and can be supplied to a display for interpretation thereby as is well understood in the art.